Getting a Continuous Understanding of Trial Users

ConnectWise  |   4 min read   |   2017-2019

Note: The information in this project is my own and does not necessarily reflect the views of ConnectWise.

ConnectWise Control is a core product of the ConnectWise suite used for remotely controlling computers. I wanted to gain insights of Control's trial experience. There was no easily accessible and consistent data at the time that gave such qualitative insights, so I wanted to fill in this knowledge gap. Some knowledge I was seeking was:

- Why users signed up for a trial.

- What their job was.

- What they needed Control for.

- What they liked and disliked about Control.

Goals

One of the main goals of this project was to engage in fast continuous user research, it was not meant to answer one question and be done. Additionally, I wanted to understand who prospects were and what their experience with ConnectWise Control was. I hoped to consistently provide a way of seeing if the product was heading in the right direction and could continue to grow. As time passed, I saw this research as a potential way to onboard new employees to ConnectWise too. They could use this research to hopefully understand the types of users that trial Control.

Users

I targeted any user trialing Control. There was no other specific age range, particular behavior, characteristic I was going after.

My Role & Team

No one asked for this research, I led this initiative. I came up with the research script, conducted interviews while taking notes, and summarized the notes. I collaborated with a product marketing manager to come up with an in-app message for recruiting users. The message went to users on their fourth day of their trial.

Recruiting

It was initially challenging to recruit users to talk to. I started out by manually emailing users based off a list I would get from the Data team and offered no incentive. This got very little responses. I offered no incentive first because I was hesitant in asking for it. I had the most success using the in-app message offering some incentive. This message was created through an outside app purchased later on.

Research Method

I chose interviews because I was seeking answers to questions along the lines of why and what. My approach was to treat these interviews as open-ended conversations. As users were from all over the United States and places like the UK and Australia, I did these interviews remotely. I used the following questions below for my script:

- Please give a little background on your job. How long have you been doing that for?

- Why did you decide to sign up for a trial?

- Describe your team structure/team size.

- What activities take up most of your day in your job?

- What are some challenges in your job?

- How do you define success in your job?

- Describe your workspace.

- What other apps do you use in your daily work?

- Example scenarios for using Control

- How has your experience been with Control? ( What went well, what didn’t go well?)

- Anything else to add?

Documenting Data

I initially documented raw data through Powerpoint slides. Later on, I switched to documenting in Confluence when I had access to it. With Confluence, I could create a research template allowing me to summarize interviews much faster.

Sharing Data

Rather than sharing findings through presenting a standard slide deck, I wanted to come up with a hands-on and creative way. My office had about 24 employees total, so I saw this as a great chance to get everyone on the Control team involved. Inspired by this article , I took random quotes from my interviews and had the team represent quotes through drawings. I did this exercise twice.

I was also asked to share some of my research to my UX team and in a Control business unit meeting that included people from departments such as Marketing. Near the end of 2018, I created an infographic for the Control team giving a high level summary of the research done for the year, a sort of "end of year insights."

Impact

I conducted 103 interviews and had support from managers to do this research. As mentioned in the previous section, I engaged with different teams sharing my findings.

Retrospective

Show and Tell

I wish I would have asked users more on showing how they used Control so I could have gotten an even clearer picture of their experience.

Involving Others

I wish I would have asked other colleagues to listen in on some interviews to further instill a user-first mindset. Examples being product managers and developers.

Sharing Feedback Faster

I didn’t share feedback as consistent and fast as I could have despite looking into different approaches. I think sharing this feedback faster could have had impact on Control's roadmap. I wish I would have asked for some advice on how to do this.

Next Project